Device Having a Camera Used to Detect Visual Cues that Activate a Function of the Device

ABSTRACT

Methods and apparatuses are disclosed that allow an electronic device to autonomously adapt one or more user alerts of the electronic device. For example, some embodiments may include a method for operating a haptic device including driving a haptic device using a control signal, measuring a frequency related to the operation of the haptic device and comparing the measured frequency with a target frequency. A control signal is adjusted based on the comparison to drive the haptic device to the target frequency.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 17/351,091, filed Jun. 17, 2021, which is a continuation ofU.S. patent application Ser. No. 16/584,661, filed Sep. 26, 2019, nowU.S. Pat. No. 11,043,088, which is a continuation of U.S. patentapplication Ser. No. 15/897,968, filed Feb. 15, 2018, now U.S. Pat. No.10,475,300, which is a continuation of U.S. patent application Ser. No.15/583,938, filed May 1, 2017, now U.S. Pat. No. 9,934,661, which is acontinuation of U.S. patent application Ser. No. 14/942,521, filed Nov.16, 2015, now U.S. Pat. No. 9,640,048, which is a continuation of U.S.patent application Ser. No. 14/512,927, filed Oct. 13, 2014, now U.S.Pat. No. 9,202,355, which is a divisional of U.S. patent applicationSer. No. 13/943,639, filed Jul. 16, 2013, now U.S. Pat. No. 8,860,562,which is a continuation of U.S. patent application Ser. No. 12/750,054,filed on Mar. 30, 2010, now U.S. Pat. No. 8,487,759, which is acontinuation-in-part of U.S. patent application Ser. No. 12/571,326,filed on Sep. 30, 2009, now U.S. Pat. No. 8,552,859, the contents ofwhich are incorporated by reference as if fully disclosed herein.

TECHNICAL FIELD

The present invention relates generally to haptic devices in electronicsystems, and more particularly to a self adapting haptic device.

BACKGROUND

Electronic devices are ubiquitous in society and can be found ineverything from wristwatches to computers. Many of these electronicdevices are portable and also include the ability to obtain a user'sattention through the use of an alert device. For example portableelectronic devices like cellular phones and watches contain alertdevices such as vibrating motors, speakers, and/or lights to attract theuser's attention. Because of their portable nature, many of theseportable electronic devices are made as small as possible byminiaturizing the components therein. As part of this miniaturizationeffort, the alert devices in the electronic devices are often made assmall as possible in order to conserve space. However, theseminiaturized alert devices can be problematic for several reasons.

First, these miniaturized alert devices may be inadequate to obtain theuser's attention in a variety of different situations. For example, ifthe user of a cell phone is in an environment where there is a greatdeal of ambient noise, such as a concert or live sporting event, thenthe user may be unable to see a visual alert from a miniaturized lighton the phone, hear an auditory alert from a miniaturized speaker in thephone and/or unable to detect vibration coming from the phone'sminiaturized vibration motor.

Additionally, because of electronic devices often contain slightvariations in the way they were manufactured, the actual response of thealert device within the electronic device may vary between electronicdevices. In other words, slight variations in the actual manufacturingof an electronic device may cause the electronic device to reactdifferently to the same force driving the alert device. For example, thevibration frequency may vary between phones of the same make and modelbecause of manufacturing tolerance, and therefore, the same amount ofvibration from a vibrating motor may unintentionally produce differentlevels of user alerts. Furthermore, performance variation may occur overtime due to bearing wear, dust, oxides on brushes, and/or temperaturechanges.

Thus, methods and systems that adaptively adjust the alert deviceswithin electronic devices to overcome one or more of these problems aredesirable.

SUMMARY

Methods and apparatuses are disclosed that allow an electronic device toautonomously adapt one or more user alerts of the electronic device. Forexample, some embodiments may include a method for operating a hapticdevice including driving a haptic device using a control signal,measuring a frequency related to the operation of the haptic device andcomparing the measured frequency with a target frequency. A controlsignal is adjusted based on the comparison to drive the haptic device tothe target frequency.

Other embodiments may include an electronic device that autonomouslyadjusts at least one operating parameter of a haptic device. Theelectronic device includes a haptic device and a sensor configured tomonitor the haptic device during operation of the haptic device. Afeedback loop is provided that includes a filter coupled to the sensorand an error detector coupled to the filter, wherein the error detectoris configured to compare a measured signal with a target signal togenerate an error signal. A controller configured to receive the errorsignal and adjust a control signal in response to the error signal toachieve a desired operational parameter is also provided.

Still other embodiments may include a method of adjusting user alerts inan electronic device. The method including initiating operation of ahaptic device by overdriving a control signal provided to the hapticdevice and actively braking a motor of the haptic device to stopoperation of the haptic device

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an electronic device capable of self adapting one ormore of its alert devices to obtain the attention of a user in differentenvironments.

FIG. 2 illustrates one operating environment for the electronic device.

FIG. 3 illustrates an alternate operating environment for the electronicdevice.

FIG. 4 illustrates an alternate embodiment of an electronic device thatincludes a plurality of motors.

FIG. 5 illustrates a block diagram of an electronic device capable ofself adapting one or more of its alert devices to obtain the attentionof a user in different environments.

FIG. 6 illustrates a feedback and control system that may allow theelectronic device to achieve a target frequency that is customized tothe current operating environment.

FIG. 7 illustrates a control signal that may be generated by thefeedback and control system shown in FIG. 6 .

FIG. 8 illustrates operations for determining a reference valuecorresponding to a maximum target frequency corresponding to a currentoperating environment of the electronic device.

FIG. 9 illustrates an electronic device with a feedback and controlsystem for adjusting operating parameters of a haptic device.

FIG. 10 is a flowchart illustrating operation of the electronic deviceof FIG. 9 in accordance with an example embodiment.

FIGS. 11-13 illustrate example torque and angular speed curves for ahaptic device.

FIGS. 14 and 15 illustrate drive signals and corresponding vibrationamplitudes for haptic devices.

The use of the same reference numerals in different drawings indicatessimilar or identical items.

DETAILED DESCRIPTION

Embodiments of electronic devices are disclosed that allow theelectronic device to autonomously observe its current operatingcondition and adjust its user alerts accordingly. The electronic devicemay determine its current operating environment (e.g., indoors,outdoors, contained in a purse or bag, etc.) through a series of sensormeasurements. Based upon these sensor measurements the electronic devicemay both select and/or optimize the user alerts to suit the currentoperating environment. For example, some embodiments may utilize thesensor measurements to determine which of the possible user alerts isbest suited to the current operating environment of the electronicdevice—e.g., if the current operating environment is indoors in aconference room, then the auditory alerts may not be the most suitableuser alert in this operating environment. Other embodiments may utilizethe sensor measurements to optimize the user alerts. For example someembodiments may include operating a motor to cause the electronic deviceto vibrate and obtain the user's attention through tactile sensation. Inthese embodiments, the sensor measurements may be utilized to activelytune the motor such that the electronic device achieves a targetfrequency that best corresponds to the current operating environment ofthe electronic device.

Although one or more of the embodiments disclosed herein may bedescribed in detail with reference to a particular electronic device,the embodiments disclosed should not be interpreted or otherwise used aslimiting the scope of the disclosure, including the claims. In addition,one skilled in the art will understand that the following descriptionhas broad application. For example, while embodiments disclosed hereinmay focus on portable electronic devices such as cell phones, it shouldbe appreciated that the concepts disclosed herein equally apply to otherportable electronic devices such as the IPOD brand portable music playerfrom Apple Inc. In addition, it should be appreciated that the conceptsdisclosed herein may equally apply to non-portable electronic devices,such as computer equipment (keyboard, mice, etc.) and/or gaming devices(e.g., gaming controllers). Furthermore, while embodiments disclosedherein may focus on optimizing the vibration output of the electronicdevices, the concepts disclosed herein equally apply to other forms ofuser alerts, such as sound devices and/or light devices. Accordingly,the discussion of any embodiment is meant only to be exemplary and isnot intended to suggest that the scope of the disclosure, including theclaims, is limited to these embodiments.

FIG. 1 illustrates an electronic device 100 capable of autonomouslyadjusting one or more of its alert devices to obtain the attention of auser of the electronic device 100 in different environments. For thesake of discussion, the electronic device 100 is shown in FIG. 1 as acell phone, such as an IPHONE brand cell phone from Apple Inc. Theelectronic device 100 may include one or more alert devices capable ofobtaining the attention of the user of the electronic device 100,including a vibration motor 102, a light source 104, and/or a speaker106. FIG. 1 also shows that these alert devices 102, 104, and 106 may becoupled to one or more sensors 108 and 110 located within the electronicdevice 100. As will be discussed in greater detail below, the sensors108 and 110 in the electronic device 100 may include devices thatmeasure indications about the environment in which the electronic device100 is operating. These measurements may include the movement, proximityto the user, location, whether the user is holding the electronic device100, ambient light levels, and/or ambient noise levels experienced bythe electronic device 100 to name just a few.

In some embodiments, the sensors 108 and 110 may be configured toprovide a primary functionality, such as receiving user or environmentalinput related to applications or programs running on the device. Thesesensors may be repurposed or additionally used to provide secondaryfunctionality for the device. “Secondary functionality” generally refersto the use of one or more sensors for an operation, or to provide inputor output, other than their primary purpose. Thus, a temperature sensorconfigured to monitor the heat of a casing may also be used to detect arise in heat from the presence of a user's hand as “secondaryfunctionality.”

As another example of secondary functionality, sensor(s) may be used todetermine the operating parameters of haptic devices. As a more specificexample, measurements from an accelerometer are often primarily used todetermine an orientation of the device 100. However, in some instances,the signals outputted by the accelerometer may be used with interactivesoftware (such as a video game) to provide an additional input devicefor user gameplay, thereby providing secondary functionality for theaccelerometer. Continuing this example, the accelerometer may berepurposed for determining the operation of a haptic device. Forexample, when the haptic device operates, the accelerometer may be usedto indirectly measure the operating parameters (such as frequency) ofthe haptic device to determine whether there is degradation in thehaptic feedback. The accelerometer may compare the range of motion ofthe haptic device during operation to a stored profile to determine ifthe haptic feedback is too great or too weak. A feedback control loopmay be provided to correct for any deviance from a determined operatingrange, as described in detail below.

Based these measurements, the electronic device 100 may autonomouslydecide the most effective way to obtain the user's attention in thatparticular environment. FIGS. 2 and 3 illustrate two distinct operatingenvironments for the electronic device 100, where the alert used toobtain the user's attention may vary between these two operatingenvironments. Referring first to the operating environment shown in FIG.2 , the electronic device 100 may be lying flat on a table 200 such asmay be the case when the user is in a classroom or meeting. If thesensors 108 and 110 are implemented as an accelerometer and microphonerespectively, then the electronic device 100 may detect that it is in aclassroom or meeting by the sensors 108 and 110 reporting no movementfrom the accelerometer and/or a relatively low ambient noise level fromthe microphone. Upon detecting that it is operating in this environment,the electronic device 100 may silence any audible alerts to the user,such as when there is an incoming phone call.

Conversely, FIG. 3 illustrates a user 300 carrying the electronic device100 in a purse 305 where it may be jostled around. If the sensors 108and 110 are implemented as an accelerometer and an ambient light sensor(ALS) respectively, then the electronic device 100 in this operatingenvironment may detect that it is in a confined space that is dark bythe ALS reporting a relatively low ambient light level and that theelectronic device 100 is being moved around by the accelerometerreporting movement. This operating environment may require louder useralerts than the situation shown in FIG. 2 , for example, the strength ofuser alerts, both auditory and vibrations, may be increased in thesesituations.

Referring again to the electronic device 100 shown in FIG. 1 , the motor102 shown includes an eccentric weight 112 coupled to a motor body 114via a shaft 116. When an electric signal, such as a voltage signal, isapplied to the motor body 114, the shaft 116 begins to rotate causingthe weight 112 to move in a substantially orbital path. Because theweight 112 is uneven, as the weight 112 begins to be rotated in thissubstantially orbital path, the motor 102 begins to vibrate, and as aresult, the motor 102 causes the entire electronic device 100 tovibrate. When the electronic device 100 is deployed in differentoperating environments, the maximum target frequency of the electronicdevice 100, or frequency at which the entire electronic device 100experiences its maximum vibration, may vary between different operatingenvironments. For example, comparing the two operating environmentsshown in FIGS. 2 and 3 , the electronic device 100 making physicalcontact with the table 200 will have a different target frequency thanthe same electronic device 100 being jostled around in the purse 305. Bymonitoring the sensors 108 and 110 based upon these measured parameters,the target frequency of the electronic device in these differentoperating environments may be determined. Furthermore, by activelyadjusting the vibration of the motor 102 based upon these measuredparameters, the electronic device 100 may be adjusted to achieve thistarget frequency in different operating environments. That is, theelectronic device 100 may actively “tune” itself to its target frequencyusing measurements obtained from the sensors 108 and 110 and adjustingthe motor 102. In the embodiments where the electronic device 100 is aphone, this active adjustment may occur within the period of a singlering of the phone, such that the phone is ringing at its targetfrequency before the end of the first ring of an incoming call tomaximize the chances of obtaining the user's attention. Similarly, whenthe electronic device 100 is a multi-function device that includes theability to check electronic mail, this active adjustment may occurwithin the period of time it takes to notify the user of a new mailevent.

FIG. 4 illustrates an alternate embodiment of an electronic device 400,which includes a plurality of motors 402-408 coupled to the sensors 409and 410. As shown, in this embodiment, the plurality of motors 402-408may be in different locations within the electronic device 400 so as tovibrate different portions of the electronic device 400. In thisembodiment, the target frequency of the electronic device 400 may beachieved by actuating the plurality of motors 402-408 in differentpatterns, where the pattern of actuating the plurality of motors 402-408varies according to the different operating environments of theelectronic device 400. For example, if the electronic device 400 islocated within the purse 305 as shown in FIG. 3 and the sensors 409 and410 indicate that one end 412 of the electronic device is touching thebottom of the purse 305 and the other end 414 is not touching the bottomof the purse 305, then the motors 402 and 408 may be actuated to achievethe target frequency of the electronic device 400 while the other motorsin the plurality 404 and 406 are not actuated. Thus, the electronicdevice 400 may be tuned to its target frequency in differentenvironments by selectively actuating one or more of the motors withinthe plurality of motors 402-408.

FIG. 5 illustrates a block diagram of an electronic device 500 that maybe employed in the embodiments shown above. As shown, the electronicdevice 500 includes a plurality of sensors 502-512 that couple to aprocessor 516. These sensors 502-512 may be used alone or in combinationto determine the current operating environment of the electronic device500. The microprocessor 516 may be further coupled to one or more alertdevices 518-522.

As was mentioned above, the ALS 502 senses the ambient light of theenvironment that the electronic device 500 is in and reports thisinformation to the processor 516. When the processor 516 receives thisambient light information, it can modify alert operations of theelectronic device 500 accordingly. Thus, in the embodiments where theelectronic device 500 is a phone, if ambient light measurements indicatethat the level of ambient light is relatively high, then alertmechanisms other than the light 518 may be used to obtain the user'sattention, such as the motor 520 and/or speaker 522, because the light518 may be unperceivable to the user because the ambient lightconditions. As was mentioned above, the information from the sensors maybe combined such that the ambient light measurement from the ALS 502 maybe used in conjunction with other measurements, such as ambient noiselevel, to detect a current operating environment of the electronicdevice 500.

The microphone 504 may sample the ambient noise level of the environmentthat the electronic device 500 is in and report this information to theprocessor 516. Thus, the microphone 504 may indicate that the ambientnoise level is too high for the speaker 522 to obtain the user'sattention, and therefore, alert mechanisms other than the speaker 522may be used to obtain the user's attention, such as the motor 520 and/orthe light 518. In the embodiments where the electronic device 500 is aphone, then the microphone 504 may be the microphone used by the user ofthe electronic device 500 when using the phone.

The infrared (IR) detector 506 may detect a user's proximity to theelectronic device 500 and report this information to the processor 516.In some embodiments, the IR detector 506 may include one or more solidstate sensors, such as pyroelectric materials, which detect heat from auser's body being near the electronic device 500. In other embodiments,the IR sensor may include a light emitting diode (LED) that emitsinfrared light which bounces off a user in close proximity to theelectronic device 500 and is detected by an IR sensor that is based upona charge coupled device (CCD), where the CCD may detect reflected IRlight emitted by the LEDs. In still other embodiments, a photoresistormay be used in place of or in conjunction with the CCD. Regardless ofthe actual implementation of the IR detector 506, the IR detector 506may convey its signal to the processor 516 as an indication of a user'spresence near the electronic device 500, and this indication may be usedin conjunction with one or more of the other sensors to determine thecurrent operating environment of the electronic device 500.

The camera 508 may capture certain visual cues for use in determiningthe operating environment of the electronic device 500. In someembodiments, the camera 508 may be integrated within the ALS 502. Inother embodiments, the camera 508 may be located on a separate portionof the electronic device 500 and may be used to confirm measurementsfrom one of the other sensors, such as the ALS 502. For example, in theevent that the electronic device 500 is implemented as a phone and theALS 502 is positioned on one side of the phone, such as the face sidethat the user positions against their head when using the phone, and thecamera 508 is positioned on the opposite side of the electronic device500 as the ALS 502, then the camera 508 may be used to confirmmeasurements indicating that the phone is in a certain operatingenvironment.

Furthermore, in some embodiments, measurements from the camera 508 maybe used to provide additional information regarding the operatingenvironment of the electronic device 500. For example, if the electronicdevice 500 is implemented as the phone shown in FIG. 2 , where the phoneis lying face down, and the ALS 502 is located on the face of the phonewhile the camera 508 is located on the opposite side of the phone, thenby the ALS 502 indicating that it is receiving substantially no lightwhile the camera 508 indicates that it is receiving light, then mayindicate that the phone is lying face down on the table.

The accelerometer 510 may indicate the general orientation of theelectronic device 500. In some embodiments, this indication may bethrough measurement of a damped mass on an integrated circuit, such as amicro electro-mechanical system (MEMS) For example, the accelerometer510 may include one or more “in-plane” MEMS accelerometers, which aresensitive in a plane that is parallel to the sensing element (such asthe damped mass), and therefore multiple dimension (such as two andthree dimension accelerometers) may be formed by combining two or morein-plane accelerometers orthogonal to each other. Other embodiments mayutilize out-of-plane MEMS accelerometers, which are sensitive topositional movements in a direction that is in a plane that isperpendicular to the sensing element (sometimes referred to as Coriolismovement). Some embodiments may combine one or more in-plane MEMSsensors with one or more out-of-plane MEMS sensors to form theaccelerometer 510. As mentioned above, the accelerometer 510 may be usedto determine orientation of the electronic device 500 (such as face up,face down, tilted, etc.) and/or whether the electronic device 500 isbeing jostled about by the user (such as inside of the purse 305 shownin FIG. 3 ). By providing the measurements from the accelerometer 510 tothe processor 516 in addition to measurements from other sensors, theprocessor 516 may combine the measurements and confirm of the othersensors. For example, if the combination of the ALS 502 and the camera508 indicate that the electronic device 500 is lying face down (asdiscussed above with regard to FIG. 2 ), then the processor 516 mayutilize measurements from the accelerometer 510 to confirm thispositional information.

The global positioning system (GPS) sensor 511 may indicate the positionof the electronic device 500 with respect to the latitude and longitudecoordinates of the Earth as determined by signals from a plurality ofgeosynchronous satellites orbiting the Earth. Since the GPS sensor 511may be unable to receive satellite signals while indoors, the GPS sensor511 may be used to detect whether the electronic device 500 is indoorsor outdoors, and the processor 516 may adjust the alerts accordingly.

The capacitive screen sensor 512 may detect whether the user is makingcontact with the electronic device 500, and/or how much contact the useris making with the electronic device. For example, if the user isholding the electronic device 500 in their pocket, then the capacitivescreen sensor 512 may indicate a certain capacitance level associatedwith the user's body. On the other hand, in the event that theelectronic device 500 is located the purse 305 as shown in FIG. 3 , thenthe capacitive screen sensor 512 may indicate a different capacitanceassociated with the fabric of the purse 305. Also, when the capacitivescreen sensor 512 senses substantially no capacitance value, then theelectronic device 500 may be on a table 200 as shown in FIG. 2 .

Table 1 illustrates how values from the capacitive screen sensor 512 maybe confirmed by the other sensors, such as the ALS 502. For example,when the ALS indicates that the ambient light level is low, such as whenthe phone may be in a pocket or in the purse 305, then the capacitivescreen sensor 512 may be consulted by the processor 516 to determine ifthe capacitance value corresponds to human versus non-human capacitanceso that the processor 516 may determine the operating environment anadjust the user alerts accordingly. Similarly, in the event that thecapacitive screen sensor 512 indicates that substantially no capacitanceis measured, then the ALS 502 may be consulted to determine if the lightlevel is high indicating that the operating environment is on the table200 in a bright room or, if the light level is low, indicating that theoperating environment is on the table 200 in a dark room, such as anight stand. The processor 516 then may adjust the alerts accordingly,such as by silencing alerts from the speaker 522 in the event that theelectronic device 500 is on a night stand.

TABLE 1 ALS 502 High Low Capacitive Screen Full screen, In pocket Sensor512 human Full screen, In purse non-human Nothing On conference table Onnight-stand

Referring still to FIG. 5 , each of the sensors 502-512 may be used bythe processor to optimize the performance of the light 518, the motor520 and/or the speaker 522 to the operating environment of theelectronic device 500. FIG. 6 depicts a block diagram of an illustrativefeedback and control system 600 that may be implemented by theelectronic device 500 to control the motor 520 such that its movementallows the electronic device 500 to achieve a target frequency that iscustomized to the operating environment. As shown in block 605 of FIG. 6, the control system 600 may include a storage unit 605 that includes areference value that is reported to other items in the control system600. For the sake of discussion, this disclosure will discuss thereference value as based upon an accelerometer measurement, although itshould be appreciated that this measurement may be based upon a widevariety of sensors, such as one or more of the sensors 502-512. Also,the reference value in the storage unit 605 may be a combination ofmeasurements from more than one of the sensors 502-512.

The control system 600 may include an error detector 610 coupled to thestorage unit 605 and the accelerometer 510. The accelerometer 510 mayreport its measurements to the error detector 610 in the same form asthe reference measurements stored in the storage unit 605. As wasmentioned above, measurements from the accelerometer 510 may representmovement of the electronic device 500 in the current operatingenvironment of the electronic device 500, and as a result, themeasurements from the accelerometer 510 may be used to measure thetarget frequency of the electronic device 500. During operation, theerror detector 610 may compare the reference value stored in the storageunit 605 with the current measurement from the accelerometer 510 andoutput an error signal E_(s).

The error detector 610 may couple to a motor controller 615 and therebyprovide the error signal E_(s) to the controller 615. The controller 615may utilize the error signal E_(s) in controlling the input signals tothe motor 520, such as by generating a control signal that isproportional to the difference between the reference value stored in thestorage unit 605 and the accelerometer 510. As mentioned above, theelectrical signal applied to the motor 520 may be a voltage, andtherefore, the control signal generated by the motor controller 615 mayvary one or more aspects of the voltage that is applied to the motor520. For example, control of the motor 520 may be accomplished byvarying the amplitude, frequency, and/or duty cycle of the voltage thatis applied to the motor 520.

In some embodiments, the motor 520 may be controlled using a pulse widthmodulated (PWM) signal. This PWM signal may allow more robust control ofthe motor 520 than conventional methods, such as an on/off control. Inthese embodiments, the PWM signal may be used to initially overdrive themotor 520 to reduce the rise time or ‘spin up’ for the motor 520 therebyproducing a sharper turn on of the motor 520. Similarly, in theseembodiments, the PWM signal may be used to underdrive the motor 520, orinductively brake the motor 520, so as to achieve a sharper turn off ofthe motor 520. This sharper on and off action of the motor 520 mayresult in more noticeable tactile sensations to a user when using themotor 520 as an alert device.

FIG. 7 illustrates varying the frequency of the control signal where thefrequency varies with respect to time. Note that the varying frequencymay be monotonically increasing during each cycle of the control system600 (section 705), unchanged during each cycle of the control system 600(section 708), monotonically decreasing during each iteration of thecontrol system 600 (section 710), or be dithered between two or morevalues during each cycle of the control system 600 (section 715).

Referring back to the control system 600 shown in FIG. 6 in conjunctionwith the electronic device 500 shown in FIG. 5 , in some embodiments,the storage unit 605, error detector 610, and motor controller 615 maybe incorporated into the microprocessor 516. Thus, during operation, themicroprocessor 516 may sample values from the accelerometer 510 (whichrepresents movement of the electronic device 500 within its currentoperating environment) and actively control the motor 520 such that theerror signal E_(s) is minimized and the reference value stored in thestorage unit 605 is achieved. The reference value that is stored in thestorage unit 605 may be modified autonomously by the electronic deviceso that the control system 600 is actively tuning itself to thischanging reference value. By changing the reference value stored in thestorage unit 605, and tracking the measurements from the accelerometer510 in response to this varying reference value, the target frequency ofthe electronic device 500 in its current operating environment may becalculated. For example, as the reference value is varied, the referencevalue that causes the electronic device 500 to achieve maximum resonancein the current operating environment (as measured by the accelerometer510), may be stored in the storage unit 605.

FIG. 8 illustrates operations 800 for determining a reference valuecorresponding to a target frequency of the electronic device. The targetfrequency of the electronic device may be a resonant frequency of theelectronic device 500 in its current operating environment, oralternatively, may be a frequency of the device that maximizes a user'sperception of the alert. It should be appreciated that the operationsshown in FIG. 8 are illustrative, and that other operations fordetermining a reference value may be performed in other embodiments. Theoperations 800 are discussed herein in the context of the electronicdevice 500 being a phone that is receiving an incoming call, however,the operations 800 may be applied in other contexts, such as in thecontext of a personal digital assistant (PDA) alerting a user to anappointment for example.

Referring now to FIG. 8 , block 805 shows the electronic device 500receiving an incoming call. Generally, the duration of a single ring foran incoming call may be five seconds and the phone may ring for a totalof five rings before being transferred to voicemail, or twenty fiveseconds. In some embodiments, the operations 800 may be triggered whenthe electronic device 500 beings to ring on the first ring and completewithin this first ring, and therefore the block 805 occur on first ring.In other embodiments, the operations 800 may occur on a subsequent ringand complete within that subsequent, and therefore the block 805 may bea subsequent ring. In still other embodiments, the operations 800 maybegin at the beginning of the first ring and complete before the phonetransfers the call to voicemail.

Once the electronic device 500 receives an incoming call, the electronicdevice 500 will detect the current system state per block 810. Forexample, the microprocessor 516 may observe the values of one or more ofthe sensors 502-512 to determine their values, and as was discussedabove, based upon one or more of these measurements, the electronicdevice 500 may predict the operating environment of the electronicdevice (e.g., on a table as shown in FIG. 2 versus in the purse 305 asshown in FIG. 3 ).

Next, in block 815, the initial reference value may be loaded into thestorage unit 605. The initial reference value to be stored maycorrespond to an initial estimation of the reference value that matchesthe current operating environment. For example, momentarily to FIGS. 3and 6 , if the processor 516 determines that the phone is in the purse305, then the processor 516 may consult a lookup table to determine apredetermined reference value to be stored in the storage unit 605 suchthat the initial target frequency achieved by the control system 600generally corresponds to the phone being located in the purse 305. Thisinitial target frequency stored in the storage unit 605 may be optimizedby subsequent operations.

Referring back to FIG. 8 , block 820 includes a decision block todetermine whether the initial reference value is to be optimized. In theevent that no optimization is desired, such as when the control system600 determines that the initial reference value achieves a targetfrequency that is within a threshold of a predetermined maximum targetfrequency, then control may flow to block 825, where the motor 520 maybe actuated corresponding to the initial reference value.

On the other hand, in the event that the block 820 determines thatoptimization is desired, then a dithering process may be utilized todetermine the target frequency of the electronic device 500. Thisdithering process may begin in block 830 where the control signalprovided to the motor 520 may be increased, for example, by increasingthe frequency as illustrated in the section 705 of FIG. 7 . In block835, each time the control signal is increased by the controller 615,this value may be stored for determination of the target frequency ofthe electronic device 500. Next, in block 840 the control signalprovided to the motor 520 may be decreased, for example, by decreasingthe frequency with the controller 615 as illustrated in the section 710of FIG. 7 . In block 845, each time the control signal is decreased,this value may be stored for determination of the target frequency ofthe electronic device 500.

Next, in block 850, the microprocessor 516 may compare the values storedin blocks 835 and 845 and adjust the reference value in the storage unit605 accordingly. For example, if the value stored during block 835 isgreater than the value stored during block 845, then increasing thecontrol signal per block 830 may result in the electronic device 500getting closer to its target frequency than decreasing the controlsignal per block 840. Thus, the controller 615 may increase thefrequency of the control signal to the motor 520 by increasing thereference value stored in the storage unit 605 per block 855 and thencontrol may flow back to block 830 where the dithering process beginsagain.

Likewise, if the value stored during block 845 is greater than the valuestored during block 835, then decreasing the control signal per block840 may result in the electronic device 500 getting closer to its targetfrequency than increasing the control signal per block 830. Thus, thecontroller 615 may decrease the frequency of the control signal to themotor 520 by increasing the reference value stored in the storage unit605 per block 860 and then control may flow back to block 830 where thedithering process begins again.

The dithering operations shown in blocks 830-845 are merely illustrativeof the operations that may be implemented in determining the maximumtarget frequency of the electronic device 500 in its current operatingenvironment and the operations 800 shown in FIG. 8 may vary in otherembodiments. For example, in some embodiments, there may be adisproportionate number of increases (block 830) in the control signalcompared to decreases (block 840) in the control signal or vice versa.Also, in some embodiments, instead of modifying the frequency of thecontrol signal, other portions of the control signal, such as the dutycycle or amplitude of the voltage, may be modified during the ditheringprocess.

In still other embodiments, the maximum target frequency may bedetermined by stepping through reference values incrementally. Forexample, the reference value stored in the storage unit 605 may besubstantially zero (e.g., on the order of several hertz) and thisreference value may be stepped up from this initial value to a maximumreference value. As this reference value is stepped and the controlsystem 600 reacts to this changing reference value, the measurement ofthe accelerometer 510 may be stored by the processor 516 in order tofind a maximum target frequency of the electronic device 500. Bystepping through a range of reference values in this manner, theprocessor 516 may determine if there are multiple harmonic targetfrequencies in the target frequency spectrum of the electronic device500 and determine which of these harmonics produces the largest targetfrequency of the electronic device 500.

Because one or more characteristics of the motor 520 may vary as afunction of temperature (e.g., the electrical resistance of windings inthe motor may increase with temperature), wear (e.g., the brushes thatcommutate the windings in the motor 520 may have an increasing theelectrical resistance over time), and/or friction (e.g., the internalbearing structures of the motor 520 may have an increase in the amountof friction over time, causing the motor to spin more slowly in responseto applied voltage). These characteristics may include macro scalechanges due to aging and wear and/or micro scale changes due totemporary heating in a hot car or due to the generation of heat in themotor windings during operation. Using one or more of the aboveidentified methods, the motor 520 may be operated in such a manner so asto counteract one or more of these effects. For example, using a PWMcontrol signal, in conjunction with measurements from the one or moresensors, changes in performance of the motor 520 as a function of timemay be compensated for. Such measurements could be inferred indirectlyfrom measurements of the armature resistance of the motor 520 (e.g., tocompensate for temperature/brush wear) or directly from measurements ofmotor speed at a known duty cycle (e.g., using the accelerometer 510).In addition, while these degradations in performance may be compensatedfor, they may also be used to trigger a repair or diagnostic history tobe communicated to the user, or to the manufacturer or seller of thedevice.

FIG. 9 illustrates an example electronic device 900 having a feedbackloop for controlling the operating parameters of a haptic device. Theelectronic device 900 may include any or all of a storage device 902, anerror detector 904, a motor controller 906, a motor 908, a sensor 910and a filter 912, as shown, as well as other components and/or devices.The motor controller 906 may utilize an error signal provided from theerror detector 904 to control the operating signals provided to themotor 908. In particular, the motor controller 906 may adjust thefrequency, amplitude and/or duty cycle of a PWM control signal tocontrol the operating parameters of the motor 908.

Turning to FIG. 10 , a flowchart 920 illustrating operation of theelectronic device 900 in accordance with the embodiment of FIG. 9 isshown. Generally, the flowchart 920 relates to using an accelerometer tosense vibrations of a haptic device. However, it should be appreciatedthat the same or similar steps to those shown in the flowchart 920 maybe implemented with other sensors and other haptic (or other output)devices to achieve a desired level of control for such devices. Forexample, thermocouples, gyroscopes, compasses, and so on may be used tomonitor or sense parameters related to the operation of a motor used ina fan or a hard drive and provide a feedback signal. In someembodiments, the measurements may be taken directly while in otherembodiments, indirect measurements may be taken. That is, it should beappreciated that in some embodiments, effects of the operation of themotor is measured (i.e., the vibration from the motor) rather than theactual operation parameters. For the purposes of this discussion,however, the term “operating parameters refers” to measurements relatedto the operation of the motors and is not exclusive to either theeffects of operation or the actual operation parameters.

In some embodiments, one or more sensors may be repurposed from aprimary purpose, or additionally used, to sense the operation of themotor. For example, an accelerometer may be repurposed to determine theoperating frequency of a haptic device. That is, measurements from anaccelerometer may generally be used to determine an orientation of thedevice 100 and/or may be used with interactive software, such as a videogame, to provide an additional input device for user gameplay as primarypurposes. Upon actuation of a haptic element, the accelerometer may berepurposed to measure the operating parameters of the haptic element,such as the amount of vibration induced in the device 100 by the hapticelement. As such, it should be appreciated that a sensor(s) alreadyprovided with a particular electronic device may be used to monitor theoperation of a haptic element.

Returning to FIG. 10 , a PWM control signal is provided from thecontroller 906 to the motor 908 to drive the motor (Block 922). Asvoltage is provided to the motor 908 via the PWM control signal, currentrises and drives the motor which results in a vibration/accelerationoutput that may be sensed by a user. The operation of the motor is alsosensed by sensor 910 to generate a measured signal (Block 924). Themeasured signal is then processed (Block 926). In one embodiment, anoutput of the sensor 910 is filtered with a bandpass or notch filter 912to allow vibrations having frequencies near the target operatingfrequency of the haptic element to be passed through for furtherprocessing, thus eliminating acceleration measurements unrelated to themotor (Block 928). Peaks within the filtered signal are found (Block930) and the frequency of the measured signal is then determined (Block932). The finding of peaks of the filtered signal may be used todetermine a period of the measured signal. The period may then beconverted into a frequency signal, for example, for a comparison asdetailed below with respect to Block 934. Generally, if the period isdetermined to be longer than a period corresponding to the targetfrequency, it indicates that the motor is operating at a speed slowerthan the target frequency.

In some embodiments, the error detector 904 may include software,hardware and/or a combination of the two and may be configured toconvert the filtered signals from the sensor 910 and filter 912 into asignal having units indicative of an operating parameter of the motor908, such as frequency, temperature, angular velocity, and so on. Inother embodiments, discrete components other than the error detector 904may be used to convert the measured signal into units that may indicatean operating parameter for the motor 908.

The measured frequency is compared with a target frequency provided fromthe storage device 902 to the error detector 904 to generate an errorsignal (Block 934). The generated error signal is provided to the motorcontroller 906 and the control signal is adjusted according to the errorsignal (Block 936). In one embodiment, a duty cycle of a PWM controlsignal may be adjusted by the motor controller 906 to achieve the targetfrequency. For example, to increase the current in the motor armature,the duty cycle of the PWM control signal may be increased. The controlsignal is then provided to the motor 908 to drive the motor (Block 922).

In some embodiments, the motor controller 906 may store or have accessto information related to the target frequency and/or the torque andangular speed curve information so that it may appropriately adjust thecontrol signal to achieve the target frequency. As such, in someembodiments, the information accessible by the controller 906 may serveas a reference point for the operation of the haptic element todetermine changed circumstances related to the operation of the hapticelement over time, thus allowing for adjustment of the operatingparameters to achieve and/or maintain operations at or near desiredoperating parameters.

FIGS. 11-13 illustrate example torque and angular speed curves. Inparticular, FIG. 11 illustrates an example torque and angular speedcurve 1000 which may be representative for the motor 908. The verticalaxis 1002 represents the torque which may have suitable units such asinches pounds or the like, while the horizontal axis 1004 represents theangular speed which may have suitable units such as revolutions per min(RPMs) or the like. In some embodiments, the curve 1000 may be generallylinear, as illustrated, while in other embodiments the curve may benon-linear.

FIG. 12 illustrates sample torque and angular speed curves 1000, and asample pivoted curve 1010, after the motor 908 has experienced wear,aging, and/or other effects that increase the friction of the motor anddegrade the operation of the motor 908. Generally and as shown in thepivoted curve 1010, the increased friction causes the curve 1010 topivot downward from a point along the vertical axis resulting in loweroperating speeds. FIG. 13 illustrates the torque and angular speed curve1000 and a shifted curve 1020 resulting from high operatingtemperatures. As shown, the shifted curve 1020 results also in loweroperating speeds. In FIGS. 12 and 13 , the dashed lines 1012 and 1022indicate the lower speeds achieved when the motor operates at a constanttorque. The lowered speeds illustrated by the pivoted curve 1010 and theshifted curve 1020 and indicative of slower operating speeds for themotor 916 may also result in poor performance of a haptic element as itis not operating at the target frequency.

In order to achieve operation at the target frequency, the speed of themotor 916 may be increased by adjustment of the PWM control signal.Specifically, the duty cycle of the PWM control signal can be adjustedto increase the current in the armature of the motor 908 and therebyincrease the speed of the motor to achieve the target frequency. Thus,the PWM control signal allows for adjustments to be made to theoperating parameters of the motor while providing a constant voltagelevel signal and acts as a variable voltage drive without actual varyingthe voltage level.

The increased current increases the PWM cycle of the motor, and thusmoves the pivoted curve 1010 and the shifted curve 1020 so that theyreflect the original curve 1000, as indicated by arrows 1030 in FIGS. 12and 13 . It should be appreciated that the pivoted and shifted curves1010 and 1020 and the corresponding shifts due to increased current aresimply presented as examples. In other contexts, due to certainoperating conditions, the curves may be shifted and or pivoted in anopposite direction.

In addition to testing and adjusting of the operating parameters of themotor 908, periodically or at random intervals, the operating parametersmay be tested for informational purposes. That is, the operation of themotor may be audited to discover how the motor is performing. This maybe useful to a manufacturer or reseller to know how an installed base ofmotors is performing. Thus, the information related to the operation ofthe motor (i.e., the information collected by the sensor 910) may betransmitted or provided to a computer database owned, operated oraccessed by a manufacturer, for example, for informational purposes. Thetransmittal of the information may be via any suitable mode includingwired and wireless modes. Moreover, the transmittal may be passive andunnoticeable to a user of the device. In some embodiments, theinformation may be provided to a user interface of the device in whichthe haptic element is operating to inform a user of any performanceissues. This may be useful for knowing when a cooling fan is notoperating properly, for example, so that it may be fixed before a systemoverheats or to know when a hard disk drive is beginning to fail.

In the foregoing examples, it should be appreciated that the motion of adevice is measured to control a haptic element within the device. Thus,not only is the sensor (e.g., accelerometer) being used for a secondarypurpose, it also takes an indirect measurement in order to tune thehaptic (or other) device. The feedback loop may include one or moresensors and the sensors implemented may take various differentmeasurements. For example, in some embodiments, a thermocouple may beused for measuring a device temperature to infer a motor operatingtemperature. In another embodiment, a microphone may be used formeasuring a ringtone volume or quality. In some embodiments, themicrophone may also be used to determine a volume for a hard disk drivewhen spinning. In some embodiments, a gyroscope may be used to determineacceleration of a device when a vibrating haptic element is actuated.

In some embodiments, the ramp up and stopping of motors may be improved.FIGS. 14 and 15 illustrate drive control curves with correspondingvibration amplitudes. Specifically, FIG. 14 illustrates a traditionalon/off drive control signal 1400 for the motor 908 with voltage in thevertical axis and time in the horizontal axis. A corresponding vibrationamplitude curve 1402 is illustrated below the traditional drive controlsignal. The vibration amplitude has a sawtooth form 1404 because themechanical time constant of the vibration motor may be long with respectto the input signal, resulting in a slow rise time and a “soft” feel totransition between on an off vibration.

In contrast, FIG. 15 illustrates a drive control curve 1500 and acorresponding vibration amplitude curve 1506 achievable using PWMcontrol signals. As illustrated, the drive control curve 1500 isoverdriven in the rise 1502 and in the spin down 1504, resulting incrisper rise time in the vibration amplitude 1508 and in the vibrationspin down 1510 and 1512. Generally, the rise time can be overdriven in aPWM control signal by increasing the duty cycle of the signal. The spindown time after an one signal can be reduced by shorting the leads ofthe motor to generate an inductive braking effect on the motor or byapplying an opposite polarity to the leads to actively brake the motor.These techniques provide a crisper, more noticeable transient betweenthe on and off states of a vibrating alert device.

Although concepts have been presented in relation to specificembodiments, it should be appreciated, that the concepts may beapplicable over a number embodiments not specifically described hereinbut falling within the scope of the present disclosure. Accordingly,embodiments disclosed herein are not to be construed as limiting.

We claim:
 1. A smartphone, comprising: a housing at least partiallydefining an interior cavity; a screen coupled to the housing; a camerapositioned within the interior cavity to detect visual cues and output asignal indicative of the visual cues; and a processor configured to:receive the signal from the camera; identify, using the signal, aparameter of an operating environment; and in response to identifyingthe parameter of the operating environment, activate a function of thesmartphone.
 2. The smartphone of claim 1, wherein: identifying theparameter of the operating environment comprises determining that a useris contacting the smartphone.
 3. The smartphone of claim 1, wherein:identifying the parameter of the operating environment comprisesdetermining that the screen is contacting a surface.
 4. The smartphoneof claim 1, wherein: activating the function of the smartphone comprisesproviding a user alert.
 5. The smartphone of claim 1, furthercomprising: a haptic device within the interior cavity, the processorconfigured to adjust operation of the haptic device based on theidentified parameter of the operating environment.
 6. The smartphone ofclaim 5, wherein: adjusting operation of the haptic device comprisesadjusting a type of haptic output provided by the haptic device.
 7. Thesmartphone of claim 5, wherein: adjusting operation of the haptic devicecomprises adjusting a strength of haptic output of the haptic device. 8.The smartphone of claim 1, further comprising: an ambient light sensorpositioned within the interior cavity to detect an ambient light leveland output an ambient light level signal indicative of the ambient lightlevel, the processor configured to receive the ambient light levelsignal and identify the parameter of the operating environment using thesignal and the ambient light level signal.
 9. The smartphone of claim 8,wherein: the camera is positioned within the interior cavity to detectthe visual cues from a first side of the smartphone; and the ambientlight sensor is positioned within the interior cavity to detect theambient light level from a second side of the smartphone opposite thefirst side.
 10. The smartphone of claim 1, further comprising: one ormore additional sensors positioned within the interior cavity to measureone or more indications of the operating environment and provide one ormore additional signals indicative of the one or more indications, theprocessor configured to receive the one or more additional signals andidentify the parameter of the operating environment using the signal andthe one or more additional signals.
 11. A method for operating anelectronic device, comprising: detecting one or more visual cues using acamera of the electronic device; identifying, based on the one or morevisual cues, a parameter of an operating environment; and in response toidentifying the parameter of the operating environment, activating afunction of the electronic device.
 12. The method of claim 11, wherein:identifying the parameter of the operating environment comprisesdetermining that a user is contacting the electronic device.
 13. Themethod of claim 11, wherein: identifying the parameter of the operatingenvironment comprises determining that the a screen of the electronicdevice is contacting a surface.
 14. The method of claim 11, wherein:activating the function of the electronic device comprises providing auser alert.
 15. The method of claim 11, further comprising: adjustingoperation of a haptic device of the electronic device based on theidentified parameter of the operating environment.
 16. The method ofclaim 15, wherein: adjusting operation of the haptic device comprisesadjusting a type of haptic output provided by the haptic device.
 17. Themethod of claim 15, wherein: adjusting operation of the haptic devicecomprises adjusting a strength of haptic output of the haptic device.18. The method of claim 11, further comprising: measuring an ambientlight level using an ambient light sensor of the electronic device,wherein the parameter of the operating environment is determined basedon the one or more visual cues and the ambient light level.
 19. Themethod of claim 18, wherein: the camera is positioned on the electronicdevice to detect the visual cues from a first side of the electronicdevice; and the ambient light sensor is positioned on the electronicdevice to measure the ambient light level from a second side of theelectronic device opposite the first side.
 20. The method of claim 11,further comprising: measuring one or more indications of the operatingenvironment using one or more additional sensors of the electronicdevice, wherein the parameter of the operating environment is determinedbased on the one or more visual cues and the one or more indications.